An Intonational Phrase Boundary and Pitch Accent Dependent Speech Recognizer

Does prosody help word recognition? In this paper, we propose a novel probabilistic framework in which word and phoneme are dependent on prosody in a way that improves word recognition. We describe the idea of prosody dependent speech recognition by building a prosody dependent speech recognizer that conditions word and phoneme models on two important prosodic variables: intonational phrase boundary and pitch accent. It is known that intonational phrase boundaries induce salient lengthening to the phrase-final speech units, while pitch accents induce distinct pitch variation on the accented syllables. Effective prosody-discriminative hidden Markov models (HMMs) can be built by conditioning on prosody only a small subset of HMM distributions: the duration PDFs and the acoustic-prosodic observation PDFs. The prosody dependence of the acoustic-phonetic observation PDFs is ignored in our investigation, resulting in a prosody dependent recognizer that has a good trade-off between performance and parameterization. To accurately model the duration of the prosody dependent allophonic models, explicit duration hidden Markov model (EDHMM) is used for both training and decoding. A new acoustic-prosodic feature, transformed from the normalized F0 contour by an artificial neural network (ANN), is incorporated into the acoustic feature vector for the acoustic modeling of accent induced pitch variation. This prosody dependent speech recognizer is able to improve word recognition accuracy by an absolute 1.8% over prosody independent recognizers on the Boston University Radio News Corpus which is prosodically transcribed using ToBI labeling system.